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~ The MAILING DA TE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )I3 Responsive to connmunication(s) filed on 21 May 2001 . 
2a)n This action is FINAL. 2b)S This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G, 213. 

Disposition of Claims 

4) ^ Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-28 is/are reiected. 

7) 0 Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner, 

10)^ The drawing(s) filed on 31 May 2000 is/are: a)K accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
1 1 )n The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1. D Certified copies of the priority documents have been received. 

2. p Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachmeht(s) 

1 ) !3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) S Infomnatlon Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Informal Patent Application (PTO-1 52) 

Paper No(s)/Mail Date 4 and 5 . 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 
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DETAILED ACTION 

1 . This action is responsive to communications: Application filed on 5/3 1/00 and 
Information Disclosure Statements filed on 9/14/00 and 5/21/01. 

Information Disclosure Statement 

2 . The information disclosure statements (IDS) submitted on 9/14/00 and 5/21/01 have been 
considered by the Examiner. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-4, 8, 12-13, 16-21, 23-25, and 27-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Mukhopadhyay et al. ("Passive Capture and Structuring of 
Lectures", 10/1999, ACM Multimedia '99, pages 1-11 (submitted in the IDS filed on 
9/14/00)) in view of Mitchell et al. (USPN 5,963,966 - issued on 10/1999). 

Regarding independent claim 1 and (dependent claims 13, 23, 25, and 27), Mukhopadhyay 
discloses: 

A method for linking a document to a video (Mukhopadhyay on page 2, right column teaches 
slides are synchronized with the video; on page 3, left column teaches slides are matched to the 
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video; and on page 10, left column teaches inserting digitized output of scan converters when 
demonstrations are given), comprising the steps of: 

document identifier (Mukhopadhyay on page 5, see figure 7 teaches each slide image are 
set with a value to match with each video frame); 

obtaining a video file having a plurality of video frames (Mukhopadhyay on page 5 
teaches video sequence of a plurality of segments); 

transforming the plurality of video frames into plurality of respective video frame 
identifiers (Mukhopadhyay on page 5 teaches matching a representative frame from each 
segment in T to one of the slides in S; a video sequence comprises a set of time values T ); 

comparing the document identifier with the plurality of video frame identifiers 
(Mukhopadhyay on page 5 teaches computing a video sequence of v(t) segments or frames to 
match against every slide image S (see figure 7)); and 

linking the document to a first video frame in the plurality of video frames 
(Mukhopadhyay on page 5, see figure 7 teaches matching slides in each frame to synchronize 
video data to the slides). 

However, Mukhopadhyay does not explicitly disclose "obtaining a scanned document 
having margins", "removing margins from scanned documents", "scaling the scarmed document" 
and "transforming the scanned document". 

Mitchell discloses "obtaining a scanned document having margins", "removing margins 
from scanned documents" and "scaling the scanned document" on col. 8, lines 36-43 and col. 9, 
lines 18-19 teaches document pages are scanned into images; wherein the images are cropped to 
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remove the borders and rescaled; "transforming the scanned document" on col. 6, lines 36-40 
teaches translating from paper to electronic form such as SGML, HTML, and text. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 2, Mitchell discloses scanning a document to create an image on 
col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 3, Mitchell discloses cropping the image from the scanned 
document to remove the borders on col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
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paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 4, Mukhopadhyay discloses a reduced image of the sHde displayed 
on the Lecture Browser user interface to fit the corresponding video within the user interface (see 
figure 2). 

Regarding dependent claims 8 and 19, Mukhopadhyay discloses using a camera to record the 
presentation on page 2. 

Regarding dependent claims 12 and 28, Mukhopadhyay discloses comparing color histograms 
of successive frames with the slides on page 5, right column - page 6, left column. 
Regarding dependent claim 16, Mitchell discloses document images contain hyperlinks on col. 
8, lines 60-67. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Intemet or a media such a 
disk or CD-ROM. 

Regarding independent claims 17 and 21, Mukhopadhyay discloses "link between a document 
file and a segment of a video file responsive to the comparison of a transformed document and 
transformed video frame", on page 5, see figure 7 teaches matching slides in each frame to 
synchronize video data to the slides). 

However, Mukhopadhyay does not explicitly disclose "a scanned document". 
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Mitchell discloses "a scanned document" on col. 8, lines 36-43 and col. 9, lines 18-19 
teaches document pages are scanned into images. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 18, Mitchell discloses "a scanner to create scanned document from 
a physical document", on col. 8, lines 36-43 and col. 9, lines 18-19 teaches document pages are 
scanned into images using OCR. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 20, Mukhopadhyay discloses a Lecture Browser user interface for 
viewing slides and video on page 2, see figure 2. 

Regarding dependent claim 24, Mitchell discloses removing borders from the scanned 
document on col. 8, lines 36-43. 
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It would have been obvious to a person of ordinary skill in the art at the time the 
invention v^as made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

5. Claims 5-7, 9-11, and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Kumar (USPN 5,835,129 - issued on 11/1998). 

Regarding dependent claims 5-6, 9-10, and 22, Mukhopadhyay and Mitchell disclose the 
invention substantially as claimed as described supra. However, Mukhopadhyay and Mitchell 
do not explicitly disclose "orthonormal transform" and "discrete cosine transform". 

Kumar discloses transformation of the image frame and the process of dividing image 
frames into blocks using DCT (discrete cosine transform) defined by the orthonormal basis on 
col. 8, line 60 - col. 9, line 3. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Kumar into Mukhopadhyay and Mitchell to provide the 
transformation of an image using DCT defined by the orthonormal basis, as taught by Kumar, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to facilitate the division of 
image frames into blocks. 
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Regarding dependent claims 7 and 11, Mitchell discloses removing borders from the scanned 
document on col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay and image of Kumar, in order to provide a low-cost and high performance 
solution for converting paper into a form that can be accessed through the network such as the 
Internet or a media such a disk or CD-ROM. 

6. Claims 14-15 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Bloomberg (USPN 5,181,255 - issued on 01/1993). 

Regarding dependent claims 14-15 and 26, Mukhopadhyay and Mitchell disclose the invention 
substantially as claimed as described supra. However, Mukhopadhyay and Mitchell do not 
explicitly disclose "extracting handwritten annotations from scanned documents". 

Bloomberg discloses extracting handwritten annotations from a scanned image on col. 1, 
lines 24-33, see Abstract. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Bloomberg into Mukhopadhyay and Mitchell to provide a 
way to extract handwritten annotations from a scanned image, as taught by Bloomberg, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to provide an improved 
system to identify and separate printed text and handwritten annotations from an image. 
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Conclusion 



1. 



The prior art made of record and not relied upon is considered pertinent to applicant's 



disclosure. 



Horace Ho-Shing IP, et al. "Hypertext-Assisted Video Indexing and Content-based 
Retrieval", 1997, Hypertext 97 (ACM), pages 232-233 discloses mapping text source to a video 
shot (see figure 2). 



examiner should be directed to Almari Yuan whose telephone number is 703-305-5945. The 
examiner can normally be reached on Mondays - Fridays (8:30am - 5:00pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Feild, can be reached on 703-305-9792. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



8. 



Any inquiry concerning this communication or earlier communications from the 



/JOSEPH FEILD 
SUPERVISORY PATENT EXAMINER 




AY 

March 20, 2004 



